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{54} RSullM^meta, muiti-feed and interacUve virtmi [ftsettfon syetems and methods 



(57) A rrulti-csnnerg:, rriuSi-iSec virltiai inssnion sys- 
tem an<i tnoihod for fepiacing lar gs:s in the camsra out- 
puts w)!h images hjss: a switcher {31 ! for fscsiving the 
csmera ouipul from each c' a plurality of carnetas >;iO) 
md for providing a ocrrbined video ou;put and iaiiy sig- 
nsfs synchtonr^eci !o ins co-named video ouiput to indi- 
cat3 \m& camsra or cairsarss provide ihs ccmsined 
vtd«& output 6S any giysn time; 8 pSytaffty of UacKing 
modylss {S48'54!;',, each fiecesving *a camsfa otJiput 
iroma camgfSi arsei for tracksns the iocatfen of the tsr^t 
in trx! camerg owsput; and g tfansttion anstysis mocfuis: 
(50) for receiving the taity signals trom the switehsr sncf 
tor oioviding infefmaiieri reprciing tfis identitlcalion of 
and tr^nsft icins between camsfas in the combinsd video 
outpyt. A scenario ser^-sr dsltnes scsipls for each feed 
Each graphic tssd rtcdute (5S1-563) among ssvstsi 
modiiies provides; s fftsd by smserttng images imo the 
oonitinoci video oiitpus based on target tracking data, 
csmora ideniiticstiGii ;;r,c< iransitior. infoiroation, arid 
script ;n!:jr;ri.:irr:-.'; Fof fricdiiving the scripts in responss 
•o .r.tC'taciivttv t)y vie^-ve; active areas are pnsvided to 
fi'.e v!o>ver whici" SiC- r;£:rt oi Sns camem output and 
irackcd !fi >!-!S same way as ii^e targets. 
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fOOOii "lU- pfosjns iavertiiof; rciascs to yinuaS sns^ttico systems io> video transmission anci, mote particuSarSy, te 
iysiof'-is -r-exiixis sof proYidifig vijiuai siissrtion mio s vidso SransiTifesion sy«ems ts-sai uses rrs-iiUpfo camerssancf 
p.-C'", d;:s 'fu.'Lip!c output feeds and/br for providing virtiisi inserticri in fssponse to ioteraciion item & v'lt'mr. 
[0002: y,fr..a: rscao'-. systems afi3 systc-rns shs! reptsics or overlay s targat tcgion iri s vidso sequence with a sab- 
stttu'o .mago fljapt.ic of vigso A typsea! vrrtuai insertion system taKes as «s input a video feed (gensraiiy a "diit/" feed 
iv vn' ;roiuc;C:S grsnrvc iaysis arja'csr sp&cial sfiects) siicwmg, e.g., s sporiirtg svent and repress a corr-nfisrcisi biSijoard 
iiix;-.'." in I'-.Q •.■ ceo --jad aavGriistrtf) orui prcdust or comparty with & Virtual grsphic ad^/snising aioti^&r produd or 
jcm ;;;;iy, Tft-s r,-od feics ts i\-,en bsoadcasl or othoY/tse deliverad to the viewers. 

[0003] ;n genera!, vsriuai im^^r]ion syslsrvs .-dsritiJy \hB target region to be rgpiaced in the vsieo fsea either by image 
rccogn-uor; technicucs: or by ,';n,-iSy..;inc; d':!c Uom an instf-jft^snJ&dciamefs ptwitJingthstssd. VJstya! irssertior! systorrss 
a;so ciiior ,n -rcr Uaw :y ssQvcnz.i^g. n panic jiar, the focatiori tirna and place of the itTtsge pfoco$ssng systems and 
vir-.t.ai ;nssriiO!-s riev.cs .A muili cart^ra -(Ddhak'' virtiiat it^sertion sysfem is dIsclcssU snc' ciaimed in appiicaliori PCTj' 
FRS9«'3iaa herfttnastsr 'the PCT ApjJtolicn* 

IQQ04] k fnors cornpisiB description to ths feastc corn;X;tients and Dfcx.sssgs of virii;a; insertion systems is provsisd 
m tro^PCT appiicatic-^, U S. Palsnt N>os, 5,2S-i.933. 5.363.392. 5,-'bc »7 ' S'-a f ;.-43.85S and Ftmch Patent No 
34-C5B35, to whioi's tsfBrence may iji? mads. .A tracking rfsxiuie assccstec \V:;r; each Kamera outpu! provides )n!0.''ma' 

[00051 A S'c;c;«!-!g ;iiodu;e is m&ociaisd vm ma camera tor provicincj jiifcrmation on tne tarpet recston if ffiere is a 
f oqii:fe;Tier:i for N c»iiiersnt output ieeds vvs-n drfierent subsiitiJiiO'ns ^o- difi'^ren; audic^css, irer^ N dif srant subsSituiion 
rrioduies nsao be Droytdg<i. 1 f thers are U cameras at>-ieix;: whic!- a '-elcct^o-^ rr-ay be n-<s:'? ^ control rccrr; far deiivery 
of H output iesds, Ih&r. U H insertion rnoduies ars r^ece'ispjy and t^o 

[0006] Th„o-v^i n-ericn . crcnut.'OsLv .c* - v c t .,p s ii m . -t- - H<i 1 ns3non systsm 
s.iT.iiar 1o that rfisclosoG tn the i-'C i appitcatto.- a.-c n: .sas =ido,:ri-a. .rod^jics for penoiming certain functbns v^tiich 
rs^suit in a mui!)-osrrisr&, rnuSt^-ieod virtual snser-sor: system nsv;ng prsceteed -scrtpi* tor contfolirng im arefer, eontent 
Slid nrRing or i-f-^sprtions during and 2v«m. Advaniageotisiy iitese prs»d«n:Red scrip 

•he svfrnt. sjithfir das&dcn :ntgractivity with the visivsr a en other iactcrs, sucn as irspu ^cnr Sioaccastcr 
imO?l For ihat pufposs. thers ts provised a systen as ds?ift&d in clam 1 snd a prc:-;.-iS3 acccrdiny lo •if^aim 7. Oiher 
!0£!ur3s are definso i.i the oliier claims, irj & partictjfar etniJodijTiQnt. each camera has its '.-.e -j o! v^aw o; a-, iive svent. 
|000S| The system is based on a concurfsnt processing arcliitscture and an tmportan? etemant a; ths systems iS s 
rr-cenario sms'. Such .-a se;vsr Ciars dofrne scripts and trarisJef tespectivs scripts to a graainc ieed jT!odu!c vh:ch is 
ci3s<*;iated ivish ssch oi tf^e multipicj ouftput vidao tssds. The sesoam sarvsr asso controis the Saw of data between 
the: graphic feed modyi^s and the vanc-ys cmsr woduies ft»t form pars oi the sysJsm. system is quite esfrtpiex, 
((3009] Each trscktng modtiie tiaoKs the tsrgets by eitiier p«;<sessing ]m image received f rH5fT5 the cawras {typicaity 
.i ^titty* feed v^^ich asr:aady mciudss gre^iisGSs ste.) or stmlyaing date provided by ifistfijtrteftlatiari on ths camom (m., 
pan. ttit, i:corfi, Socu-s), or a c^pnjbmtkjrt for ihe two. T.hjs iracksng is gytomatic artd aigoriShmic fccfcsuee it must be 
Sicco'ispiisiicd i.n icai o; near rea! time. The trackirsg irjforraatbn is pnovided to the graphic to^ aicduies as wait 
Si* so i.-.? scenario scivcr it sriay bs triuch simpfiii-sd whsn using the fixed fisid csmsm approach described in EP 

fOGi{sj vnc !.art;-,tc- .:-ri;.;iY3i3 fncaiiic i-^iov co^; ;n-orrt^5:ti-2n regarding the idenJitication of the earners or cameras 
' - ^ - 2 " - \ r iico , c s'- r . -^r^ s) the transft'cn 5na!v5« sysiem bases i;s ctJtpiit or. various 

oata, niclijGing ihs tai>y S:gna=3 provided by ths swncnsr tmuitiploxsr) in a typicai outssse broadcasJsr {OB) «an, md 
vsic,. . a.w.-;.^ ,..g.3td.r,:5 Co,..•;■i.„^^!„^:>t^.v;^^.•^ vj,vc:n v,-.- d,eo:s^Cr^i -n thi: OB v.av!, .Advaritagsousiy, thssa verbal orders 



^ ^ ' ^ ^ <- t -fpi I tc-r-" r-t -ip-) a^abythstiacKtt^grnod- 

ufe.^, .1 !•_•: !d??ntf!:03tion ot the -'.:>n-aif ■ carr:ers(s} pjovioedby tr.e iransitson analysts !75cdute, andtiiescfiptfnBintbrrnjuion 
un';.«.:i-c 3 C0.1 c r i ropiaccfrieat irnagss or ar^imations;. to r?ia:-5ipaSa{s ft© spsciftec rspijicernenl frr5ages or 
-jriitTts-iof:;; ; ir;:! -.o 5: -.: :-.5 specititJCi ta.r8ets at {.he- scripted titTtes. Anirrssiiarrs typic&iiy cc^mpf ise a series of images 
wh£Cf5 Sifs ssquo;ri!;;;i:y c'tspiayed 

[0013] The 3!-chiicciu;o 3r-:ci fu:ictiO!isii;y of -he present invention piiovtdes s large irnoi^ c: *ie;cibiirty. aitowincs its 
jse o-i a diverse variety oi da!ive,-v s^'sterj^s such as TV. DTV, HDTV, Cab.e. Sass^'te ir^-c act vc -^sievisbr:. ifiiemci, 
etc. t^is pras irji -tcr Jtu< \'it\< i? ued v. -t.T ,t i 5 osco ><.rt" the o-vebp rc s-tisri^ jt on syst«ns xvhich 3>|cns 



ior disUiijuisci bfoadcastirsg, ihal is, bioadcastu'ig io an identiriabie ssgi-nsnJ ct vt«v*srs, sucn as ihro^ict- casie Thg 
mulUpto fssd capatoility o? tiio pressns invsnEicr! aiJov.'s she various outp-Jt ioods -o be tailored to a par^ciifar subset of 
viewers, sisch as ijy tocatiori or by language, and with a high ciegree o; granuiafity. For exampls,, 3 cable fead may fae 
djffei gntia5ec!atlhsarea, coimy, zip cede, hsadsncf, crpscfeste! levsS. Ts-ius. tiie advsrtissmsnis. otc. which are insortsd 
intoa p3rticifiar brosdcasii can be rnafket (aiiored sift-ipb/ by d^anaing ihe script for the appropnaie iBs6. 
[001 4] I«,^orsov0r, th& system of the prsser.t invcftto is idea^Ey suiissd ior ose vvilh dslivsry sys-ems atoing inierac- 
Sfvtiy with iha yiswer. Sudi ddivery sys5srr:3 include digitai broadcas; over privsSs fsssworks of i'he interest. Spectficaiiy. 
ins bancivvidlh provided csbie iriiern«{ access even cifrrensiy afbvvs for ?atfly good quality digisai video jrsrtsmission 
Siod :.he <!va.!ab:e cuaiity is imprctfiMcj cDn3}an% Wiih cs&fe tntemet acc&$s, interactwiiy is mado possibSs by ti,vo-way 
oab;!5 modems or tsiephcsie-fsturn cab;e modems, inSeractivity is atso possifete wiJh oSisr dsiiv&fy systems, such as 

[OOIS] As noted s&ovs, shs niuitiplG feed capabiiUy oi the present invortlion aitows Ehe scripts to tse changed to fit 
parttaular dsiiver/ d-taractof istics, such as rsgions or ianguagas. Additional the $.cf!pis mgy diarsgesJ Of mocfifisci 
;n response 10 ;'-ti;!;i;;;<.',;y hen « vis a'sf or a ciustor ct viewers, thus snabting tsfgeisd snd intsraetive nsari<eling (s. 

g, , ons-on-one rfiarSajf ng. pcRT^issicn {T^rtc&ring. feia-ionsh^p rnarketingl 

[0016] For itirsi purDcso, lha iraeking module may bs arFaiQed fcr gariarating "hanotss" indtcative o5 tS^e ioestioft sf 
the i&fge; region and of the fact it constitutes an -aosive area", vvhscj- aro displayed on ihe tntersetiws receiver d a 
ViSWrjr I ;ier: "■•icr.5ci;vi-y i-nay t;e scriioved Py providing a cofiipiiicr dstectec ssiscsiort of m active ares, rsspon«ivs 
to a inotise ciic!-; in the asea for insfeince, and causing a change in !hs substituikxi. 

[0017] Ctrsr feaJurea a:-id advanirigos oi tne Dresenr irivention v/iii bscoms appsmnt from the detaiied descriptiof^ 

Of M:)-boJtm€t5t& Oi ;ho mventiOrt^b'i'nicn iolfcsvs retsrs ;o tne araw-nqs where-r? ■ 

vsrstbrs. 

10019] Fiyyr.3 2 a sc.hejnstic bSccii diagram of the sr-sertion systern otFigyr.s 1 in css^iuRciiorj VviiJTacsbiaijroadcasl 

sViHcm, 

fOOSOJ i" ir;,;! ■:; :3 ir> a iron! vie* of a scccfifl dispiay showiRg the dss oi "active ersss" is accoi-danee witn aftolhsf 

|002i| Figure a fiont v'iswof a scrarjn display showing the isss of 'acjivo areas" in accardance mih an aiternate 

err^bisuimsn; oi ii-e irtvsrtiicrs. 

[CK}2t;] rjguro 5 is a'-sother schesT-atizsd fepreseniatior? c; ;in mser-ion sysiom. 

[0025] .A. systsrti; h accordance wsSh sn smboain^snl 0! the invsnllon is sham in Figure 1. Portions 0? figurs 1 are 
simitar -o the "midlinK ' vsrtuai stssrtion syslssi cfecJoasd in Rgeris 6 of the PCI appiiealioo. which indudss a ryiyiti- 
csiTsera inaersi&n system. 

[0024] in FiQUfij 1. the cameras lOa. lOfc aftd 10c are instrurnsrited cameras ar-d hav-c- correspondinc; cats ih^e^ 
40s, 40b and Ado. mimht^s camsras are shoxvn. the system cai^ i:o configured to uiiii/s mpu- ffom any mmbet or 
oamsrsss. Triasisi oabSos 42a 42b arsd 42c. which carty cat^ioia powsr aaci imaoe sigDais, ar-j cotineci'sd iro-ni 
rsspcoiive cssT-sras to ths m^sijiic- cofttroi room or vsin 30, A ccrtventionst switcher 3t (jypicaSly a dtgitai svviSchSfJ is 
c.o.':r-:cc!Ed to conv-fWiicrsai video fSqu^ment 32 which, &$ cfiscussed in the PCT appisfiticn. adds whatevor grspiifcs 
and spiciai eil'cetis ihsit are to be ac&lsd Tfie conuoi roorii X aisc irjciudss a graphic- processor interfaos of GPi 36 
wh.di !i conn^ci^: fo ihc video equipmsnl 32 and whscj-. produces ouspiii ssgr^sis based on jfie feyers of graphics as 
as. spisei;,;: ei;;c.::ts ;«,g sioiv ntGlicfi} which have he&n added to ths video imago sigrisf. Tttese GPi signals are 
ssr^t ox'sr a (■:;-■>• It;.: a? 10 p^ ^nput fo 8 transitsoT ar^aiysis moduls (TAy) 50 !n addition, tafiy siga^is for cameras I0a. 
1 Ois inc ; Cc ;)re s;en; ;c TAVt 5?: frofr swi-d^er over r&spectivs cuspui iinss 4Ss, 4Sb s-f^d 4Sc; 
[0025] Video Equiprnon; 32, a-, c^coperaircn With frvviichsr 3t , producoS: on output Sine 4?, a broadcast rOffty feed'') 
ssgnai or seed, ; 0 , and cdjied signal wnich contains xvt-sstevor iayers of gfaphics and spsciaf sffscts that have been 
add3!,i to the video ;mse« stgn,=ii This dirty feed >s trsnsrTsitiod in cigita; from 50 the Graphic Feed fvtocJ-Ji^s. (QfUs). in 
" ^ h ' ' . --f5 \ \ < < - A* :5 1 5S2and56 3, respsctivefy 

^ ^ H - i u seo s-cn r3rro^jei'^5 

U^ck"-roH^e,.(., ..r,u :po--.dcv}t"=lc^ - c I 5 C "'J z Jr^icXit^ut^BsT i^l^^^^^^^ 
dSxSfrnji^s exactly fiow !0 manjpuiate enr; ins9rt«5nimaJicri insertions inSo the iseo. Far exairipis, if ttie teed contains a 
vios^D sequence -A-ferain ca-^nera lOa ti^-.,n:5!tions -o ca,me.;?i tCb c ■ .; ; ■•o-^ =aii to ..gh; .?nd if a first tsirgst is irj the 
nefci cf view of camera lOa, the inscr.ion ihai wili replace thai ;,-:;c: -.^.jr^i Ps manipuiatsd by the assoctatsd Graphic 
hsed M(sdu!e so as to rriatch the wipe as the iirst tsrg-st is wined from itss sorson. 



[00271 ■^u to t-^rnu i< y IS c-v>j 

Pt. 1 cDicatcsn T^^'bOoifoS^ r cuUj or vara «« 



.<t!i&-.pfOJsc usiRO voice !-eccs5f!S[ion -echnofoay trifouoh mtcfoohons 

Pr- t! i fiaHoTsojui^pcude-frarofh st a-^ tnwcattot n' v ren tee * a cu i otr 

> i eoin ^esx^na i>aoovc trci itc "^e ^r-^ ! t arrr as Te oae'' nade eti dtty 

^ '^■^ < >~i3 'c p tran tjiofi types & as«of dissoKs etc hcv'taa^topafe-iid 

;fr.ii;itliuii uy Gkipiitt - ;V»auie Wb! nCW 00 S)^te»Wd 

' '^f ' ' ^ i-f <^ c"" i q r V.'- uri-ci c<3t r- tu et T t-^r ten! tr<5n«>t(ion<^ 
t f o\ <i t - f I - ^ c ( c «c -t! e-L. in t c ht - r { n ^ cv p edc i r 

h f^el f t ia & c Li po u r e^'^-i -s r c i a.i uh«t;>lede loppd 

=^ -v. 1 i.nc'Xc ^Aart qc-^L.^/ j^^iUoaco, i ur?) x v \susdto sveiop « e } <*n$iiio ts&c 
c/e reoc-cedon I jrcaccp^c s <t t-er evf fjls .sp' eatdsrifo* r?'-* tso^^ informs on =5pf vic'q ir ^^s pre enl 
"s-^a 25 p - H y t^ti A iyp c?i tr«n it o JaoSe for a trroc c^r en; s>i>tom s dscfosed - ~abte 1 



TABLE 1 




THANSmONTVFe 


LENGTH OF mANSIT! OS* 


<■ i Z 


\^'!DS iefi TO r;qn! 


SsJFrgmss 












1 " ' t u 


vVipe Tod to sscjicm 








20 Prsmos 




Disserve 


3y Frames 




\>/ci^8c!tcr* Tg» 


SO Frames 



10030} fs:^^ "Vi'ycto'^ Te^dicftlo c ans =o<^ libte vc f '^t.ogii ©n tcchrtoEog* is useJ to "it terp^ef he 
^0 3 cjnd <\T 88c»o *i hcCBvot- tHnoug mn.rv<Jhore 5^ i-o e>.snipia omrrd-aaoi "JtteS.j.e 2 wpe 

> < 1 od ^{cm itoica((t,eusoc mr ^ } re c it c«n sj'-pV bi y sd c hefc acvck^ ire 'rcnsiJ o-^ ab!e 

a-- 'a-» th(^tf--n 1 -oi a « isi a*^c^ScenafloSm'Cf C3 Ti^g Scsnerio 3 rvt. u<4esine*fai(a')0fi informt©-- 

if! b!j:!a!fi:a tne scf!Dts. 

[OOC] onm ^ ^8 s r-Mt o- - - - r ^ hio- t-^ C - c F-co "c es oS 56 2 Trd'^kno 
foe .-^-^j-^.:^ rr toir-r tv t ii.dJ «5 1 U p f-xiu a— for ofo,? d,r^ cwnai.-! da{o con 

" * t v^'^ < lod '-^ A ■-►"cruu. 1^ Of o «i - " •x:tf- c ira "s ic rcdt>!c& ^ & ^ 



; fors 



)ir i -n "o 0 ^i-qjo 1 c a. pa ) i 
B 0 h<^ occndh It o' Ihi- qotro A^npl i. J--r<ii 



SP 1871 278 M 







2' ML- 




iTdc, 1 




~ rq tB 


imaqe 2 






AnirriaUon 2 






T 90 3 


'r go i 



^ 1 "rfqc ar n o jrpt, *vo ^\ / corr& st, e> s^:^ e r mo a -mage ano 



0 lUi [ j 1^ Scsfipfio sen sr ao n a preferfi- J enbod 

c Is CO J > ^re needed 

V. at ths event 



m\ nic 



ir rr «lv 



! 3 so r"* 5 !X n CPlttCS AS O ! 

tec\ 3C ran^^Ju '"^ j4v, nust 3 o\ ids the ts >' n j ! on ii^n 

hnt o! / that h Cc-n bs v.ccN.np!!<'hcsJ i tzS i riv, w rtoo t&i \ ^ 

» I "> rpi§ '^'^an nto eo-a r t q 

1 Ti r ! h cf)!-i •p nt A'^/f n 'jgso slv sc j„t& 3f i 
lOD -] -^r nof^ n } t 

f ^^.1 -If -J >5n o t 



v^ed /at ?vto 0 
b 3 an^Ss Cf 



;srn-3nJ a; Jgfgeis by Grspftts Feed Uad- 
tifnaJioiis thsmsefees and the order and 
■ivfe s6 over ^ns oeftvotk canf^sstion. 
t^i.^'n ict w'^^.tt UKc 3d int^ of ! r [of» 'eeo'^ « n to " d jre 2 whiaH is 
f^FV f( 1 l«n«.»itoni,\« rof u r^, in cof^jtinUio i 'siU cssv^ ocrfcc t 

^ 0 s o V. OTS Gfh 1 --^D o.!si«i ytes i ojote nsft 01 c to fxctvcfo /4 

^ s iv-t GF^ _dJ 3"=^fv'3 f^cJj fftjtes he e seJ^tc- &cei%s«7S and 

r f ct ed ntc ft<s ^ ^no 7^ and t! Sbs fct-ds f fher »erit lo t-s 

'5"' iv> r re ei net ^ a o ec Tic/ cd ii^rer-iiatcu oasfiw o'^ 

1 I t TOO rtoT}r''^ne rd^qSK^^eed 



' 1 urteiin a OdS o f c 



k « f I, -^MuK --lij 
KKJJOI \V) f-iUt in 
u vsjt« rr n he ti,r 



1 j tr rtins a sr t ec 0 s o s-i i b it 
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1 ! A s<mpl»!S8 teproseRSauon oi a sysssm accotdsng lo another mp&si for the trwenlfen is given in Fsguf© S, 

bel CO jor'=' ceaT£»lOv,nN. <? 

t The an ? y n « ro of"r- oL <^ d t 

[00 o frr h6 vert fc h -jb! qc a f o f on c 

te s ac ttofi^e fnrac c o t e mi e e a 

1" e t e c ay Q on 0 pfay feO n as c fc k c 

o tf \S >^ Q-ame be 09 b oa-fc t ? a o cj -^unvf cor^p «i a a c 

TV 2 t 3 of fooao*?'^ J en occu ©cfLif y r o a i>cK f n noi, 

q o hceoDoa swrcx udod ^foco ei>- con tisuack -0 nti 

T 0 t cn "^^cantu e*'-isd h g"^e e ea h oJcrn^pcte r> fersnUct s 
{ b icn ! n fhe y f rfen Owfnen ra 1 el n and 
c n Ti c m 

'"^ ^ J r\ f {}i h6 Sua Fc x nple a ^ 
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0 c -) sicsT b c ave 

que e !« nee 

w;?:h ay possioie .choices) ars snciuded w!lh ihs fssci sftd the 53c;u3s;ad statistics are extrscSec from th& ic&a by fee 



o (.«y«. <i v,B^«r c;;cKS c;^ tne piayor fci. ;n MDure 3i duftng game ixoadijasi {era 

ssiecs^on Oi a partfcufer gctiva j.ma may .tore th. previous pt^^y in fnea^cry tor repi.^.. ai a laisr i^-na 

i A ^l. ic o r^cto ocvsa-naob 

n c Of, teL a cl ( « t,3vt e £ ) 

if' 4-] A Key 0I ofienr^ffon n « f epl nt fo c t fli vo 0^ <n 

^ ^ -^r-Yi te ailhf ea ) n t c -i^-^- 1 

f^c^ f ft. °^ so c nsf T e Jr r <^ o 0 f f-c a 
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of is-.seiiing audio intc !he fesc by using au&.a -ncdulBs afong the giaphsc feed fncdides. For example, in anothsf 
Grrifcodimoni,, an £iu& scgmcnf fs ptey<5cf Ksch time a tims a oarticute; iterrs (person, aOTertisotr.ent, etc.) ss showr: 
diirins ihe bfOEidcasi. in -slsndarci TV fcroadcasf, iha suiSio ssgmsnJ is provicfsti on Jhs sscondsry audio iracK (SAP) 
wiiich ;s norrnsHy usod for as'tsrnajs isnguages, snd is mixed mh !iig norms! pfogram sound srsck as required. Ad- 
vaniggeou-siy. as descfibsd above, audio track is provided in response io tr-teracJK'iJy &y trx; viewer. !fs addition to 
3!ir:;:.;y aclcnoA^cxiging seiectior; of an aciivssrsa, fsSational audio ssgrr-ents may aiso i30 pr ovicJeci, For exarnpie, dufjng 
b; oKdca.s; d s s\-/\ir, m&st as shovm in Figure 4. if the user ciinks 3 fTvsuse en swim \&m, -he usq! migh{ hesr an 
auQio rspresenistion of the m^tnmer's bfOrhyJhms< s.g., hearibeat 

[00511 ;s is noise S^ial ■'ho mollipit; carnsfa capabtiity o? ths present invent icrs, vMh a Jraclsii-igmodyts for eacit camera, 
ptovidsa csrtosn acdsiioriai advansjjgss. First, sines the rspiacemsnJ images or snimatiGns are ussd k a numbar of 
cfisrarfis ancles irsi- coritmuilv' oi l.i-io (.ti^sge ;s sniiarwsd, IVtereovsr. s grsat dssi ot ifiiatrrsatbn aboui ti-se eveni (or 
Ssrgo'js) car only oa sxusoiaa oy the use of .<7-!i.f!fip)e a^meras. For sxasTipie. auioiTiattc detection of osrtairt geomstrkiai 
0? ooicf features is ontv pcssibse by Enangulat.bn oromsr meitieds resuirirsc two camera pos:itfons or angles (e;g.. me 
speed of ths isadar i.n a race, ti-;s pcsii^on of -iis offside mark in a fietd event, etc.}. Furthsr, wfisn a targgtls obsffuctad 
in one y-aw, l.ne rfi:r>sin.i5 daie is avaifabic- Jissougri lire view ci anothsf camata. Additionaffy, ti^s aitributcs of fhe targets 
may be dslined differently for each mtism, thus providing more fiisxitifity ;n tfis managejnerti oi the feeds (e.g., the 
sa-ns !Hi go; can tie designated onc3 as ian c^<|ue and ones as a f rsnsoarsnt to bs tnciudsd in diiforetil scenarios on 
a given feed). 

[0DS2| Punhemo!:i, vib^e tn-3 cescrspucn above related :o repiacemen; oi ii-is targeis! is §en^ra!Sy directed toward 

^" \ ' ' ' ' ""'^^^ ^ ' ^put<.^ gir 

combinsd Witb g vidso tsed i^h the background removed} at the final broadcasts^ As shown in Figurs 5, a virluai set 
comprisiag a "osw^xw." is ccrrsbiiiedwith a video feed ccnp;i"s;;K! the r.swscasfs -s to prcvdo a complete newscast. 
This system decreases- ti^.e nacassary ijenowidtt^. i;-np!wes the !ool< of the f^nal p.rcducL a;Ki even aiimvs for eiistom- 
i-?.ai!&n oi the sKckground, For sxampie, a eust!smi26d n&vsfSfoam can "os as^ in ccmititictfon with ihe news feed in 
figurs 5 to msiks tnie sr^adeast sppaar more ioeaf. 
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iking rn-ociujes, ssid pturaiity of srapftjc rr5s>dijies ; 
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r^lOx-atiw 0 s^'O' icoctoic roc m .jar:) ? -jtpij p'c^' ^ > t-^t c - ' 

oy pfocessfng sasci instromoniati&T data. 

t • It ! 1 ct I i,-is,orr as in c acn . '■snei qm &8"- famto,! srs vsis iT'odufo d-'os lOSS Sdtd snfon-d*Kjn r ega'ctng 
f t V. HC- to'^ cJ c-nd tfarsiJiorjs bpl'^csn cams^as m spid corn > nsd oeo o-toj' oc 3ud cn <3ith8f a tnsnsitton 
->c c -0 ,g he Nrs or t'^r^s,l.->r5 csed Ly hs s^vwc^ier wren {laftsrtio^ fror ons „amera {& snot*tsr o 

*oscs Signals frofn a ffirecior 

X ^-^^ c'p ^ D-^irf ~!i Lh coo s. f-sjis.'T cvstem H (,5pi o„ 3 ai less* one target o tho ou pi.S o< sash 

f fOi d . -0 "b t 8^ .idto outpjt by ii ,irg osivveen ss-io CiT-is a oulpUs and dng Uily s)g.ias m- 
d ca! . c :M \ h(c - carx e Oi.vpjt is >ncluae2 r saio co Tb«€c /oso ou out at a givoft t<ne 

- trac tng t !t> occJ! ion of said c < Ssssi one tar gei in said aaisut of eacii and ptovidinisj at< caxtout oi tracking dats • 

- ?n=«fyz)nj. SU5 ?iiv s'^^nat ^nd pro^dng transition nfor">fito"^ tsgs'^'-Q te derti'^caKWOf arc rsnsrto'^s 

bstvvssr came ra& sn ssid ccmbineji yidso outeu? • 
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8. A nietnod as in o]&m 7. wherein said images ^ 
snedulss eadi pfcividifig one feed 

9. ■ A.meihsdas iivda;(yi 5. «,vnsfetn a s<;8Fiar(c s 

10. A msihed as i;5 elaim S, turther conifXissng 
oSiCh ot said cameras oifhaf sy processing s 



15. .A rnfithocJ gs datrrs 7, whsfsin said -ransftidri infcffrtaiio- ,s bj&,od edhs'- oi^ .a iraif, to^ Ui^e cc^^. (sik Oa'a 
rateiod to the t^^p.^ot transitfons beiweer> satcSptursi .y of c^.'re.js i oSid carb ned video cwtp^t oros- ^ofce s^anais 
from a director 

iS, .Vi ititerftctivc virttjai insertfen: sj'stem eanip rising : 

- a a^raera for providing a r^n-^!& ouiput. ■ 

- a tracking nicdL;i3 for receiving said sametB output ar.d tracking a predefined active .ares in said camera otJiput 
anr; p rov d nc, h.-snsiss .;-.dicativ's of tt:-ij Socaticf? pi said ssjlive area in ssid camsm output; 

- a s.eive; ;oi- p^ ovi-jir-g said casrj&ra output to art ii-iieractve receivsr; 

n cc;:iSi" =- • toi cotccfing fi s«!..c;iC!-i oJ said «ct-vo area sh:'>. interactive rtjceiver anci for providing a rsspcnse 

15. .<x- :>i;orac;:V3 ;n£C!t:c-n syttorr^ a? in ctaisn 12. vvherei?^ S3idcorf»xr:ef and for pravldinga response to said sgJectiw 
comprises a grap.^^ic iesd ir^cdtils and whereir^ csata related to said response Is ptxsvided ia saio graphic fs-jd modute 



by :S scenario ss-v 



14 A;-s snteractivs viriu^j trisetfjon sysrem as claim 1*0r 13, vsheresn said s 



!( pfovidss said fjandtes to said 
:v5: rsceivsr cc-nprises saio comp'jtar iof detsctl.^^ a ssisctiCft of said acim 



iatersctive receiver ai\d sj-ic intei 
ares! and tor providing .e resDonse io said ssisction. 

I*. .Ail int-..:;!.;'.i\ o v t..,i. ,t..jGr;,cn 3> Sujn-i as d : c-Ann i £, 1 3 or i •vi-ioicin s«fc response to ssiti selection conprisss 
trie 'iisp-av oi o.;:!!;Ci.;af 'i-naqe on said intu-active recssver 
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